The ready ability for a business to store, process and to transmit data is a facet of operations that a business relies upon to conduct its day-to-day activities. For businesses that increasingly depend upon data for their operations, an inability to store, process, or transmit data can hurt a business' reputation and bottom line. Businesses are therefore taking measures to improve their ability to store, process, transmit, and restore data, and to more efficiently share the resources that enable these operations.
The ever-increasing reliance on data and the computing systems that produce, process, distribute, and maintain data in its myriad forms continues to put great demands on techniques for data protection. Simple systems providing periodic backups of data have given way to more complex and sophisticated data protection schemes that take into consideration a variety of factors, including a wide variety of computing devices and platforms, numerous different types of data that must be protected, speed with which data protection operations must be executed, and flexibility demanded by today's users.
In many cases, disaster recovery involves restoring data to a point in time when the desired data was in a known and valid state. Backup schemes to ensure recoverability of data at times in the past are varied. Such schemes have traditionally included periodic full backups followed by a series of differential backups performed at intervals between the full backups. In such a manner, a data set can be restored at least to a point in time of a differential backup. Such an approach can be resource intensive as permanent records of the full and differential backups must be kept in order to ensure that one can restore a data set to a state at a particular point in time, especially to point in the distant past. Further, the process of restoring a data volume from a full and a series of differential backups can be time and resource consuming, leading to delays in making the data available to the users.
One approach to providing a less resource-intensive capacity to restore a data set to a particular prior point in time is temporal storage, also known as time-indexed storage and time-addressable storage. Temporal storage can be implemented by associating a temporal volume with a particular data set. A temporal volume maintains non-present data in addition to the data in its present state. A temporal volume maintains the history of data stored on it, thus providing a way for an application to retrieve a copy of the data at any time in the past.
Temporal volumes provide an infrastructure for maintaining and accessing temporal data. Temporal volumes can be used by applications at all levels, including file systems and database management systems. In addition, temporal volumes can also be used as building blocks for data archival, versioning, replication, and backup through integration with file system and backup products. Temporal volumes preserve temporal content so that the content can be used at a later point in time for snapshots, incremental backups, replication, restoring corrupted volumes or deleted files, etc.
In a normal volume, when data changes, the corresponding data blocks are changed in situ. In a temporal volume, when a block of data is changed, the existing block can be preserved along with some record of the time of change, and then the new data is written. Old versions of a data block are maintained even when the data block is deleted. This achieves the effect of maintaining copies of one or more states of the data in the past. This process can also be thought of as continuous versioning of the data on the disk volume, and retaining snapshots of the volume whenever it changes. Another temporal storage implementation provides the same effect of maintaining data at points in time by writing new data blocks to a separate location, associating the time with the new data blocks, and manipulating associated metadata in the temporal volume to refer to the new data blocks.
There are many possible embodiments for temporal volumes. In one embodiment, the contents of a temporal volume can be preserved using an indexing system or structure. An indexing structure can be formed using a space-optimized persistent store by allocating the storage over a cache object. A cache object is a logical storage object that gives an illusion of infinite space, while using only limited actual storage space. The cache object accomplishes this by provisioning storage on an as-needed basis. In an alternate embodiment, a temporal volume can be log-structured. In a log-structured temporal volume, data modifications are preserved in an append-only log as the data is written to data volumes. A time-index is maintained on the data log, thus permitting access to data at all points of time recorded.
In another embodiment, the temporal volume can be divided into one or more regions. A region may be anywhere from one physical block of the disk to regions of kilobytes, megabytes, gigabytes, etc. Each region can have a time stamp associated with it. Applications accessing the temporal volume can specify the time stamps associated with the regions. Alternatively, a time stamp may be specified by an application or the temporal volume manager when data is written to the temporal volume.
Time history of data in a temporal volume can be supplied and maintained in several ways, including, but not limited to: continuous checkpointing, I/O controlled checkpointing, application controlled checkpointing, and periodic checkpointing.
In continuous checkpointing, every write operation to data on the temporal volume is checkpointed and includes a time stamp generated by the temporal volume. In this manner, each change to data is maintained for subsequent access.
In I/O controlled checkpointing, an application using the temporal volume can provide checkpoints or time stamps when writing to the temporal volume. Such a time stamp can be provided to the temporal volume manager with every I/O request or alternatively only with I/O requests that modify the temporal data.
In application-controlled checkpointing, an application can issue an I/O request that specifies a time stamp for a region or the entire volume when required or desired. In application-controlled checkpointing, an I/O request can be issued that specifies a new checkpoint or timestamp within the temporal volume, rather than providing a time stamp with every write. An application can tell the temporal volume when to create a checkpoint version.
Periodic checkpointing involves performing automatic checkpointing periodically (e.g., every 10 seconds or every 10 minutes). In periodic checkpointing, a temporal volume manager provides an interface that enables periodic, temporal checkpointing at the logical device (volume) level. The temporal volume manager will periodically checkpoint or timestamp all changes to data on the volume during the set period. Should a block be modified multiple times during a particular period, only the last modification will be temporally captured via a timestamp. This saves storage space by not storing each change to data.
Temporal storage provides an ability for a user or application to access data as it was at a chosen point in time. Time becomes a dimension in storage. Such a capacity can be useful, e.g., in data mining for finding patterns and behaviors in customer data; data warehouses in which the time dimension can be stored and utilized in querying data stored in the warehouse; data archival and auditing wherein the time-indexed archives can be later used for analysis; and, data restoration to a particular time.
Temporal volumes can be provided by a temporal storage appliance. Such temporal storage appliances are typically made part of a disk volume set by host-based disk virtualization. A temporal storage appliance may include, in addition to a temporally structured volume, non-temporally structured disk volumes as an aid in mirroring and data recovery.
In addition to providing an ability to recover data, businesses are also faced with the task of managing, supporting, and providing large amounts of disk space to many distributed computers. Rather than physically distribute disks attached to the computing resources, which can create logistical difficulties in maintaining and supporting the disk volumes, businesses physically disassociate disk volumes from the compute resources. Such a disassociation may take place in a storage area network (SAN) on which the disk volumes reside and through which disk volume capacity is provided to compute resources. An application server can serve as a volume manager for one or more of these SAN resident disks to create disk aggregates called logical unit numbers (LUN).
An alternative to having a volume manager resident on an application server is to offload that task to a fabric resident switch within a storage area network. Such a virtualizing fabric switch can handle the aggregation of physical or virtual storage into logical units and can also handle tasks related to mirroring of data and linking (described below) LUNs together.
A virtualizing fabric switch will pass a write stream sent from a network node to all disks that are part of a virtual LUN (VLUN). However, a virtualizing fabric switch cannot provide temporal storage capability since creation and maintenance of temporal indexes in the I/O path is undesirable. Virtualizing fabric switches do not have a cache or the capacity to perform temporal indexing outside the I/O path.
It is therefore desired to provide the benefits of temporal storage in a storage environment that provides fabric-based virtualization.